In the development of information processing systems, relational database management programs evolved allowing the user to search, access and alter data contained in numerous different database tables by using specific fields common to all such tables.
As these database systems improved, the speed and efficiency of access to these records in the database increased and additional capability was provided. For example, more recent data processing systems began to provide support for multiple simultaneous users enabling each user to even access data concurrently at a sub-page level.
Notwithstanding such improvements, one area that remained of great concern was in providing for recovery of data, such as, after I/O or power failures, i.e., system crashes. One reason for this was the vast amount of time and money which became associated with the compilation of data resident in the database as well as the great dependence which users came to have on their increasingly vital database resources. Accordingly, a great deal of development effort was expended in attempting to solve the problems associated with such data loss.
Perhaps one of the most obvious general approaches to the problem was to provide for redundancy whereby backup copies of the data were available in the event that the database or portions thereof needed to be reconstructed due to such incomplete log writes or detected log write failures. Accordingly, several techniques were developed in the art for providing such redundancy, one of the earliest being known as shadow paging which essentially involves retaining a copy of an entire page of data while updates were made to a second copy. After the newer copy containing the changes was safely written to the permanent medium, the archival copy could thence be written over. This technique was employed for example in the database product of the IBM Corporation known commercially as SQL/DS. A survey of various systems employing this shadow copy technique may be found in "File Servers for Network Based Distribution Systems", Liba Svobodova, ACM Computing Surveys, Vol. 16, No. 4 (December 1984), pages 353-399.
Although shadow paging appeared to be a viable solution in some environments it was not without its disadvantages including the expense and space involved in maintaining such shadow copies. Accordingly, database systems began implementing the transaction recovery facility by only writing changes to database records to both the changed record and to a database recovery log. The information recorded in the database recovery log insured that changes of committed transactions were incorporated into the database state during system restart following a system failure (as well as allowing changes to database records to be reversed or undone in support of transaction rollback for uncommitted transactions).
A form of this technique became developed known as write-ahead logging wherein the protocol required that changes be written to the recovery log in the permanent file prior to being made to the actual database records themselves. One problem with such logging related to the aforementioned desirability of concurrency wherein multiple users could simultaneously access the database desirably at a sub-page level, the need for such concurrency being translated for example into a need for concurrent access to index files commonly used by database programs to provide quick and efficient access to records.
Information contained in index nodes of these index files was extremely important in providing key record information that was frequently deleted or inserted as records were deleted or inserted into the database tables, and consequently such concurrent accessibility on a sub-page level was highly desirable. A particularly important aspect of such index files was that individual fields of a record in a database might frequently logically contain data which was not kept in the record for itself but rather, (by means of a pointer or descriptor) kept in a separate file. Example of such a file is known as a long field file, wherein a long field is contained, which may have an image associated with large data set type items such as audio or image data which can be extremely valuable, thus illustrating the importance of such indexes.
With the foregoing in mind, it will be appreciated that it was desirable to provide for a database recovery system of the write-ahead logging type which nevertheless provided for such sub-page level concurrency. Systems were accordingly developed such as those described in U.S. Pat. Application Ser. No. 07/059,666, filed Jun. 8, 1987, and entitled "Method for Managing Sub-Page Concurrency Control and Partial Transaction Rollback in a Transaction-Oriented System of the Write-Ahead Logging Type", now abandoned, and refiled On Sept. 7, 1989, as pending continuation Ser. No. 07/406,186, as well as pending U.S. Pat. Application Ser. No. 07/115,146, filed Oct. 30, 1987, and entitled "Method for Concurrent Record Access Using an Index Tree", U.S. Pat. No. 4,914,569. An additional reference that discusses these index files such as those commonly configured in a B-tree structure known in the art is "Efficient Locking for Concurrent Operation on a B-Tree"by Lehman and Yao, ACM Transactions on Database Systems, Vol. 6, No. 4, (December 1981), pages 650-670, the hereinbefore noted references being incorporated herein by reference.
Notwithstanding the aforementioned advances, problems nevertheless remained in providing for effective database recovery First on restart processing of such systems, files with I/O errors were not readily detectable so as to prevent and safeguard restart operations from accessing the files with attendant data loss. Further, means were not provided for readily detecting incomplete log writes or detected log write failures in order to stop the further writing of transactions. Moreover, no effective means was provided for readily identifying such error files during restart. Additionally, rebuilding of error file indexes was by no means automatic but rather required explicit user action and invalidated access plans related to the failing index.
Accordingly, systems and methods were desired for reducing data loss due to I/O errors and power failure during non-atomic writes to disk in a transaction management system using write-ahead logging protocol. Such systems and methods were highly sought whereby I/O error on index files, including system tables, caused no data loss. Also, techniques were desired for providing automatic recovery from the errors without an explicit user action to rebuild the affected indexes. Means were desired whereby power failure during log file writes caused no data loss without the necessity for employing double writes, shadow paging or the like. It was further highly desired to provide effective means whereby I/O error on user tables had limited data loss effect to the table in error. Additionally, it was desirable to provide a technique for index file rebuilds which did not invalidate the access plans related to the index. These and other desired features not met by the prior art are provided by the subject invention as hereinafter described in greater detail.